(1) Field of the Invention
The present invention relates to a video codec apparatus capable of generating an interpolation frame in a decoding process, and more particularly, to a video codec apparatus and a method that reduce an amount of calculation for estimating a motion vector and a size of hardware that are necessary for generating an interpolation frame at an optimal cost and with the least image deterioration.
(2) Description of the Related Arts
When video data is played back, additional frames are interpolated between video frames for certain purposes. For example, a video camera plays back video data while generating and displaying interpolation frames in an extended reproduction time for the purpose of smooth reproduction in slow motion. As another example, a television with a liquid crystal display generates and displays interpolation frames in a usual reproduction time for the purpose of improving time resolution of the liquid crystal display. In other words, this achieves reproduction at a high frame rate. As another example, mobile equipment plays back video data recorded at a low frame rate while generating and displaying interpolation frames in order to achieve display equivalent with that at a normal frame rate.
FIGS. 10 and 11 show basic concepts of generation of interpolation frames. FIG. 10 illustrates generation of interpolation frames for the purpose of reproduction in slow motion. FIG. 11 illustrates generation of interpolation frames for the purpose of reproduction at a high frame rate. In these figures, frames of solid lines represent decoded frames, and frames of dashed lines interpolation frames. In FIG. 10, the reproduction in slow motion is achieved by reproducing a series of video frames with interpolation frames interpolated therebetween at the same frame rate as that for the original decoded video. In FIG. 11, the reproduction at a high frame rate is achieved by reproducing a series of video frames with interpolation frames interpolated therebetween at a frame rate twice as high as the original frame rate.
These interpolation frames are generated by calculating motion vectors from the coded frames and executing motion compensation for the coded frames, so that the decoded frames with the interpolation frames interpolated therebetween are displayed smoothly as described above.
In order to achieve such purposes, there are techniques relating to apparatuses that estimate motion vectors and generates interpolation frames. Patent Reference 1 (Japanese Unexamined Patent Application Publication No. 2005-6275) shows one example of such conventional techniques.
FIG. 12 is a block diagram that shows a configuration of a codec apparatus described in Patent Reference 1. The codec apparatus shown in FIG. 12 includes a coding apparatus 210, decoding apparatus 220, and an interpolation frame generating apparatus 230.
The coding apparatus 210 codes an input image and outputs a coded stream. The coding apparatus 210 includes a subtractor 211, an orthogonal transformation and quantization unit 212, a variable-length coding unit 213, an inverse orthogonal transformation and inverse quantization unit 214, an adder 215, a frame memory 216, and a motion vector estimation unit 217.
The decoding apparatus 220 decodes an inputted coded stream and outputs a decoded video stream resulting from the decoding to the interpolation frame generating apparatus 230.
The interpolation frame generating apparatus 230 estimates motion vectors in the decoded video stream in order to generate an interpolation frame and interpolate the interpolation frame in the video stream. The interpolation frame generating apparatus 230 includes an interpolation frame generating unit 231. Further, the interpolation frame generating apparatus 230 and the coding apparatus 210 share the frame memory 216 and the motion vector estimation unit 217.
Described below is an operation of the conventional codec apparatus in FIG. 12. First described is an operation in a coding process.
When an input image is inputted into the codec apparatus, the input image is stored in the frame memory 216. Meanwhile, for prediction coding, the subtractor 211 calculates difference between the input image and a motion compensation image, and a resulting differential image is inputted into the orthogonal transformation and quantization unit 212.
The orthogonal transformation and quantization unit 212 transforms the differential image into an orthogonal transformation coefficient, and then quantize the orthogonal transformation coefficient using a predetermined parameter. The quantized orthogonal transformation coefficient is outputted as information of quantized orthogonal transformation coefficient to the variable-length coding unit 213 and the inverse orthogonal transformation and inverse quantization unit 214. The variable-length coding unit 213 executes variable-length coding on the information of quantized orthogonal transformation coefficient and outputs resulting variable-length coded information as an output stream.
In order to generate a local decoded image for motion compensation, the inverse orthogonal transformation and inverse quantization unit 214 reconstructs the differential image by executing inverse-quantization and inverse-orthogonal-transformation on the quantized orthogonal transformation coefficient information. The differential image is added to the motion compensation image by the adder 215 and stored in the frame memory 216 as a reference image to be used for motion vector estimation.
The motion vector estimation unit 217 retrieves the input image and the reference image from the frame memory 216, estimates motion vectors, executes motion compensation, and outputs the motion compensation image.
Next described is an operation for generating an interpolation frame in a decoding process.
A coded input stream is inputted into the decoding apparatus 220. The decoding apparatus 220 decodes the input stream and outputs a decoded image resulting from the decoding to the frame memory 216 and the interpolation frame generating unit 231 of the coding unit 210.
The motion vector estimation unit 217 retrieves a plurality of the decoded images stored in the frame memory 216, estimates motion vectors to be used for generating interpolation frames, and then outputs information of the motion vectors to the interpolation frame generating unit 231.
The interpolation frame generating unit 231 generates, on the basis of the plurality of the decoded images and the information of the motion vectors, interpolation frames to be interpolated between the decoded images, and then outputs the interpolation frames as output images.
For the technique shown in Patent Reference 1, the motion vector estimation unit 217 is used both for the coding process and for the decoding and interpolation frame generating process. These processes are executed through a common process of motion vector estimation.